Information providing apparatus and information providing method using virtual sound image localization processing

ABSTRACT

An information providing apparatus including a detection unit of at least one computer, a decision unit of the at least one computer, and a sound control circuit. The detection unit detects a position and a movement direction of a moving object in an area in which an article, corresponding to information that is provided, is disposed. The decision unit decides a direction in which the information is provided to the moving object from a relationship between the position of the moving object, the movement direction of the moving object, and a position in the area of the article. The sound control circuit controls a sound output attached to the moving object to localize a sound image according to the information in the direction decided.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2017-217486, filed in Nov. 10, 2017, theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate to an information providingapparatus, and an information providing method for causing a computer tofunction as a computer as the information providing apparatus.

BACKGROUND

In the related art, there is an information providing apparatus whichdetects the position of a cart used by a shopper for carrying acommodity and displays information on the commodity displayed close tothe cart on a display device attached to the cart.

However, although the shopper using the cart knows that the commodityrelated to the displayed information is near the cart, he/she is unableto know in which direction the cart is directed.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a layout example of a storeto which at least one embodiment is applied.

FIG. 2 is a perspective view illustrating an example of a cart used inat least one embodiment.

FIG. 3 is a block diagram illustrating a main circuit arrangement of acart terminal.

FIG. 4 is a block diagram illustrating a main circuit arrangement of aninformation providing server.

FIG. 5 is an explanatory diagram of a commodity database.

FIG. 6 is an explanatory diagram of a member database.

FIG. 7 is an explanatory diagram of an additional information database.

FIG. 8 is a flowchart illustrating an order of a control processperformed by a processor of a cart terminal according to a controlprogram.

FIG. 9 is another flowchart illustrating the order of a control processperformed by the processor of the cart terminal according to the controlprogram.

DETAILED DESCRIPTION

An object to be solved by at least one embodiment described herein is toprovide an information providing apparatus that can easily identify theposition of an article related to information from the providedinformation.

In at least one embodiment, an information providing apparatus includesa detection unit, a decision unit, and a sound control unit. Thedetection unit detects a position and a movement direction of a movingobject in an area in which an article corresponding to information thatis provided is disposed. The decision unit decides a direction in whichthe information from a relationship between the position and themovement direction of the moving object, and a position in the area ofthe article is provided to the moving object. The sound control unitcontrols a sound output unit attached to the moving object to localize asound image according to the information in the direction decided by thedecision unit.

Hereinafter, at least one embodiment of an information providingapparatus that can easily identify the position of an article related toinformation from provided information will be described with referenceto the drawings. In at least one embodiment, in a store such as asupermarket, the information providing apparatus for providinginformation on a commodity to be recommended to a shopper shopping byusing a cart is exemplified. In the beginning, by using FIG. 1, thelayout of the store to which at least one embodiment is applied will bedescribed.

As illustrated in FIG. 1, a store S is divided into a sales place S1 inwhich a plurality of commodity shelves R are arranged and a cart storageplace S2 near an entrance IN. The commodities which are examples of anarticle are displayed on each commodity shelf R. A shopper who entersthrough the entrance IN takes out a cart C from the cart storage placeS2 and enters the sales place S1. Accordingly, the shopper turns aroundthe sales place S1 while operating the cart C, takes out the commodityto be purchased from the commodities displayed on the commodity shelf R,and puts the commodities in a basket of the cart C. The shopper who putsthe commodities in the basket is moved to an accounting place (cashier)(not illustrated) by operating the cart C, and performs accounting.

A transmitter B is attached to each commodity shelf R. In the presentembodiment, among four corners of the commodity shelf R, thetransmitters B are attached to a pair of corners on the same diagonal.For example, the transmitter B periodically transmits a beacon signalaccording to a short distance wireless communication standard such asBluetooth (registered trademark). A unique beacon ID is included in thebeacon signal. The transmitter B principally transmits the beacon signalwith almost the transmission intensity. The transmitter B mayintentionally change the transmission intensity according to thesurrounding environment.

Meanwhile, although details will be described later, a transmission andreception circuit 104 (see FIG. 3) for receiving the beacon signaltransmitted from the transmitter B is in the cart C, and thetransmission and reception circuit 104 of the cart C that enters to thesales place S1 receives the beacon signal transmitted from at least onetransmitter B. An attachment of the transmitter B is not limited to alayout example of the FIG. 1. As described above, the transmission andreception circuit 104 of the cart C that enters the sales place S1 mayreceive the beacon signal transmitted from at least one transmitter B.

FIG. 2 is a perspective view illustrating an example of the cart C. Thecart C in which four casters C3 are attached to a lower part of a frameC2 including a handle C1, and can be moved in front and back and leftand right directions. That is, the cart C is an example of a movingbody. A part of a frame C2 becomes a supporting member C4 for detachablysupporting a shopping cart E. The shopper puts the commodity to bepurchased from above in the shopping cart E supported to the supportingmember C4.

A cart terminal 10 as an example of the information providing apparatusis attached to the frame C2 in the vicinity of the handle C1. The cartterminal 10 includes a touch panel 107, and the shopper operating thecart C with the handle C1 attaches the touch panel at a position inwhich an image to be displayed on the touch panel 107 can be identified.

An antenna 20 of a radio frequency identification (RFID) reader 105 (seeFIG. 3) is attached to a handle C1 side of the supporting member C4. AnRFID tag is attached to each commodity to be sold at the store S. Aunique tag ID allocated for each RFID tag and a commodity code forrecognizing the commodity to which the RFID tag is attached is stored inan IC memory of the RFID tag.

The antenna 20 transmits an inquiry signal as a radio wave to the RFIDtag under the control of the RFID reader 105. The radio wave transmittedfrom the antenna 20 has directionality in a mounting direction of theshopping cart E. Accordingly, the radio wave is received with the RFIDtag attached to the commodity that puts in the shopping cart E. The RFIDtag that receives the radio wave transmits a response signal includingdata of the IC memory as the radio wave. The radio wave transmitted fromthe RFID tag is received by the antenna 20, and is output to the RFIDreader 105 as the response signal. The RFID reader 105 reads the data ofthe IC memory, that is, the tag ID and the commodity code bydemodulating the response signal.

A camera 30 that is an imaging device is attached to the opposite sideto the handle C1 of the supporting member C4. The camera 30 images themovement direction (traveling direction) of the cart C. An image of thereality space imaged by the camera 30 is displayed on the touch panel107.

FIG. 3 is a block diagram illustrating a main circuit configuration ofthe cart terminal 10. The cart terminal 10 includes a processor 101, amain memory 102, an auxiliary storage device 103, the transmission andreception circuit 104, the RFID reader 105, an image processing circuit106, the touch panel 107, a sensor group 108, a card reader 109, acommunication unit 110, a virtual sound image localization processingcircuit 111, and a system bus 112. Accordingly, the cart terminal 10connects the processor 101, the main memory 102, the auxiliary storagedevice 103, the transmission and reception circuit 104, the RFID reader105, the image processing circuit 106, the touch panel 107, the sensorgroup 108, the card reader 109, the communication unit 110, and thevirtual sound image localization processing circuit 111 directly to thesystem bus 112 or via a signal input and output circuit. In this manner,the cart terminal 10 configures a computer with the processor 101, themain memory 102, the auxiliary storage device 103, and with the systembus 112 for connecting them.

The processor 101 corresponds to the central part of the computer. Theprocessor 101 controls each unit to realize various functions as thecart terminal 10 according to an operating system and an applicationprogram.

The main memory 102 corresponds to the main storage part of thecomputer. The main memory 102 includes a non-volatile memory area and avolatile memory area. The main memory 102 stores the operating systemand the application program in the non-volatile memory area. Inaddition, there is also a case where the main memory 102 stores datarequired for performing a process to control each unit by the processor101 in the non-volatile or the volatile memory area. The main memory 102uses the volatile memory area as a work area in which data isappropriately rewritten by the processor 101.

The auxiliary storage device 103 corresponds to an auxiliary storagepart of the computer. For example, an electric erasable programmableread-only memory (EEPROM), a hard disc drive (HDD), a solid state drive(SSD), and the like are used as the auxiliary storage device 103. Theauxiliary storage device 103 stores data used for performing variousprocesses by the processor 101 and data generated from the processes bythe processor 101. There is also a case where the auxiliary storagedevice 103 stores the application program.

The transmission and reception circuit 104 receives the beacon signaltransmitted from the transmitter B, and detects the beacon ID from thebeacon signal. In addition, the transmission and reception circuit 104includes a received signal strength indication (RSSI) measurementcircuit for measuring the radio wave and reception intensity, andmeasures the radio wave and the reception intensity of the receivedbeacon signal, so-called an RSSI value. Accordingly, the transmissionand reception circuit 104 supplies the beacon ID detected from thebeacon signal and the RSSI value measured by the RSSI measurementcircuit to the processor 101 every time the beacon signal is received.

When the beacon ID and the RSSI value are received from the transmissionand reception circuit 104, the processor 101 calculates a distance rbetween the transmitter B and the cart terminal 10 specified from thebeacon ID based on the RSSI value by Equation (1). In Equation (1), “A”is the RSSI value measured at a position 1 meter away from radio signalof the cart terminal 10, and “B” is a constant of radio wave attenuation(theoretically two).RSSI(r)=A−10*B/log 10(r)  (1)

The processor 101 calculates a position of the cart terminal 10, thatis, a position in sales places of the cart C by using a single piece ora plurality of pieces of distance information r. For example, when thesingle distance information r is used, a position of the cart C iscalculated by one point positioning method. Specifically, the positionof the cart C is calculated based on the transmitter B determined thatit is the closest distance. When the plurality of pieces of distanceinformation r is used, the position of the cart C is calculated by atwo-point positioning method or a three-point positioning method. In acase of the two-point positioning method, a position obtained by a ratioof the distance information r on a line where two transmitters B areconnected to each other, is specified as the position of the cart C. Ina case of the three-point positioning method, the position of the cart Cis specified by general three-point survey method. Here, a computerwhich is the main of the processor 101 configures a detection unit thatdetects the position of the cart C which is a moving object incooperation with the transmission and reception circuit 104.

The RFID reader 105 is connected to the antenna 20. As described above,the operations of the RFID reader 105 and the antenna 20 are asdescribed above.

The image processing circuit 106 is connected to the camera 30. Theimage processing circuit 106 captures in real time an image of thereality space imaged by the camera 30, and processes the captured resultif necessary. The processor 101 displays the image or the processedimage of the reality space captured by the image processing circuit 106on the touch panel 107. Here, the computer which is the main of theprocessor 101 configures an image control unit that causes an image tobe displayed on the touch panel 107 which is an image display unitattached to the cart C in cooperation with the image processing circuit106.

The touch panel 107 functions as an input device and a display device ofthe cart terminal 10. As an operation mode, the cart terminal 10 has acommodity information display mode, a navigation mode, a recommendationmode, and the like. The touch panel 107 includes a button or a modeselection button for selecting one of the operation modes as the inputdevice. The touch panel 107 may include another function button as theinput device in addition to the mode selection button.

The commodity information display mode is a mode for displayinginformation of the commodity that puts in the shopping cart E on thetouch panel 107. As described above, the cart C includes the antenna 20of the RFID reader 105 and can read data of the RFID tag attached to thecommodity that puts in the shopping cart E. When the commodityinformation display mode is selected, the processor 101 specifies thecommodity from the data of the RFID tag, and causes commodityinformation such as the commodity name and a price to be displayed onthe touch panel 107.

The navigation mode is a mode for displaying a map of the sales place S1on the touch panel 107. As described above, the processor 101 canspecify the current position of the cart C based on the beacon signalreceived by the transmission and reception circuit 104. Therefore, whenthe navigation mode is selected, the processor 101 causes a mark or thelike indicating the current position of the cart C in the sales place S1on the map to be displayed, and informs the shopper of the currentposition.

The recommendation mode is a mode for providing the information of thecommodity to be recommended to the shopper who uses the cart C. Theinformation is provided by voice. Alternatively, the information isprovided by the voice and an image. There is also a case where theinformation is provided by only an image. When the information isprovided by the voice, the processor 101 uses a sound image localizationtechnology. That is, the processor 101 controls to hear the voice as theinformation from a direction in which the commodity shelf R displayedthe commodities recommended to the shopper is provided. When theinformation is provided as the image, the processor 101 uses augmentedreality (AR) technology. That is, the processor 101 controls asynthesized image obtained by synthesizing an image as the informationof the commodity to be recommended with the image in the reality spaceimaged by the camera 30 to be displayed on the touch panel 107. At thistime, an image of the commodity to be recommended are synthesized in adirection of the commodity shelf R on which the commodities aredisplayed.

The sensor group 108 includes a sensor used in measuring for a movementstate of the cart C using technology called as a pedestrian deadreckoning (PDR). That is, the sensor group 108 includes an accelerationsensor, a gyro sensor (angular velocity sensor), a geomagnetic sensor(electronic compass), and the like. A value measured by the sensor group108 is given to the processor 101. The processor 101 measures, based onmeasured values of each sensor, information on how much the cart C movesto what direction, that is, the movement direction (angle) and theamount of movement (distance) are measured in real time. Accordingly,the processor 101 generates data indicating a movement state (flow line)of the cart C by integrating the movement direction and the amount ofmovement. Here, the computer which is the main of the processor 101configures the detection unit that detects the movement direction of thecart C that is the moving body in cooperation with the sensor group 108.

The card reader 109 reads data recorded in a card medium such as a pointcard and a membership card. A card number including a membership codefor identifying a card owner is recorded in these card media. Theprocessor 101 detects the membership code from the card number read bythe card reader 109, and specifies the card owner from the membershipcode.

The communication unit 110 performs data communication with aninformation providing server 50 via a communication network 40. Forexample, the communication network 40 is a WiFi (registered trademark)standard wireless network, a mobile communication network, or the like.The information providing server 50 will be described below.

The virtual sound image localization processing circuit 111 connects aright speaker 60R and a left speaker 60L which are sound output unitsand a sound source 70. The right speaker 60R, the left speaker 60L, andthe sound source 70 are provided in a case of the cart terminal 10. Theright speaker 60R is provided in the vicinity of an end on the rightside as seen from the shopper holding the handle C1. The left speaker60L is provided in the vicinity of an end on the left side as seen fromthe shopper. The position of the right speaker 60R and the left speaker60L is not limited in the case of the cart terminal 10. For example, theright speaker 60R and the left speaker 60L are provided at both sides ofthe handle C1, respectively.

The virtual sound image localization processing circuit 111 performs avirtual sound image localization process for localizing a virtual soundimage at a predetermined position based on an acoustic transfer functionfrom the position of the right speaker 60R and the left speaker 60L to alistening point of the shopper operating the cart C. Since the virtualsound image localization process is well known, the description thereofwill be omitted. It is preferable that the right speaker 60R and theleft speaker 60L are speakers having the directionality. By using thespeaker having the directionality, it is possible to provide voice tohimself/herself operating the cart C.

FIG. 4 is a block diagram illustrating a main circuit configuration ofthe information providing server 50. The information providing server 50includes a processor 51, a main memory 52, an auxiliary storage device53, a communication unit 54, and a system bus 55. Accordingly, theinformation providing server 50 connects the processor 51, the mainmemory 52, the auxiliary storage device 53, and the communication unit54 directly to the system bus 55 or via the signal input and outputcircuit. In this manner, the information providing server 50 configuresa computer with the processor 51, the main memory 52, and the auxiliarystorage device 53, and with the system bus 55 for connecting them.

The processor 51 corresponds to the central part of the computer. Theprocessor 51 controls each unit to realize various functions as theinformation providing server 50 according to an operating system and anapplication program.

The main memory 52 corresponds to the main storage part of the computer.The main memory 52 includes a non-volatile memory area and a volatilememory area. The main memory 52 stores the operating system and theapplication program in the non-volatile memory area. In addition, thereis also a case where the main memory 52 stores data required forperforming a process to control each unit by the processor 51 in thenon-volatile or the volatile memory area. The main memory 52 uses thevolatile memory area as a work area in which data is appropriatelyrewritten by the processor 51.

The auxiliary storage device 53 corresponds to an auxiliary storage partof the computer. For example, the EEPROM, the HDD, the SSD, and the likeare used as the auxiliary storage device 53. The auxiliary storagedevice 53 stores data used for performing various processes by theprocessor 51 and data generated from the processes by the processor 51.There is also a case where the auxiliary storage device 53 stores theapplication program.

The communication unit 54 performs data communication to each cartterminal 10 connected via the above-described communication network 40.

The information providing server 50 such a configuration uses a part ofa storage area of the auxiliary storage device 53 as the commoditydatabase 81, the member database 82, and the additional informationdatabase 83.

FIG. 5 is an explanatory diagram of the commodity database 81. Thecommodity database 81 saves a commodity record 81R in which informationon the commodity to be sold in the store S is recorded, as many as thenumber of commodities. The commodity record 81R includes items such as acommodity code, a commodity name, a unit price, commodityclassification, and a commodity image. The commodity record 81R mayinclude other items.

FIG. 6 is an explanatory diagram of the member database 82. The memberdatabase 82 saves a member record 82R in which information on a customer(card owner) to be a member of a point card membership is recorded asmany as the number of members. The member record 82R includes items suchas a membership code, name, sex, and age. The member record 82R may beconfigured by including other items.

FIG. 7 is an explanatory diagram of the additional information database83. The additional information database 83 saves an additionalinformation record 83R in which additional information provided to theshopper operating the cart C is recorded. The additional information isinformation on the commodity to be recommended to the shopper. That is,the additional information database 83 records the additionalinformation record 83R therein as many as the number of commodities tobe recommended to the shopper. The additional information record 83Rincludes items such as a commodity code, a sales position, voiceinformation, image information, and a provision condition. It goeswithout saying that the additional information record 83R maybeconfigured by including other items.

In the additional information record 83R, the commodity code is a codefor identifying the commodity to be recommended. The commodity code isthe same as the commodity code of the commodity record 81R stored in thecommodity database 81. The sales position is information for specifyinga position of the commodity shelf R in which commodities to berecommended are displayed. The voice information is informationindicating voice of content to notify the shopper of information on thecommodity to be recommended. For example, voice having content “XXX(commodity name) recommended to customer is selling on out shelf” is setas the voice information. The image information is informationindicating the image of the commodity to be recommended. For example,the image is an appearance image of commodity. The provision conditionis information indicating a condition for providing the information ofthe commodity to be recommended to the shopper. For example, thecondition is decided from characteristics information and actioninformation of the shopper. The characteristics information is sex, age,or the like of the shopper. The action information is the information ofthe commodity already input to the shopping cart E by the shopper or amovement history in the sales place S1, that is, the flow line.

In the additional information record 83R, both the voice information andthe image information are not necessarily recorded. At least the voiceinformation may be recorded.

FIG. 8 and FIG. 9 are flowcharts illustrating an order of controlprocesses performed according to a control program by the processor 101of the cart terminal 10. The control program is recorded in the mainmemory 102 or the auxiliary storage device 103. Contents of processesillustrated in FIG. 8 and FIG. 9 and explained below are only examples,and a process procedure and process contents thereof are notparticularly limited as long as similar results can be obtained.Hereinafter, flowcharts of FIG. 8, and FIG. 9, and an operation of thecart terminal 10 attached to the cart C taken out by the shopper fromthe cart storage place S2 will be described.

The processor 101 waits for receiving the beacon signal as Act 1. Whilethe cart C is in the cart storage area, the beacon signal is notreceived in the transmission and reception circuit 104. When the shopperenters the sales place S1 by operating the cart C, the transmission andreception circuit 104 can receive at least one beacon signal transmittedfrom the transmitter B. When it is confirmed that the beacon signal isreceived in the transmission and reception circuit 104 (Act 1, YES), andthe processor 101 proceeds to Act 2.

In Act 2, the processor 101 sets the operation mode to therecommendation mode. Accordingly, the processor 101 activates the camera30 by controlling the image processing circuit 106 as Act 3, andproceeds to Act 4. By activating the camera 30, imaging starts by thecamera 30. Accordingly, the image of the reality space is taken into theimage processing circuit 106 in the movement direction of the cart C.

Here, the computer which is the main of the processor 101 configures acheck unit that checks whether or not the cart C that is a moving objectenters an inside from the sales place S1 by the process of Act 1.Accordingly, the computer switches the recommendation mode to acondition that the cart enters within the sales place S1, that is, theoperation mode of the cart terminal 10 to a mode for providinginformation, by a process of Act 2.

In Act 4, the processor 101 determines whether or not to input themembership code. When the membership code is not input (Act 4, NO), theprocessor 101 proceeds to Act 6.

When the card number of the point card or the membership card by thecard reader 109, the processor 101 detects the membership code from thecard number. When the membership code is detected (Act 4, YES), theprocessor 101 proceeds to Act 5.

In Act 5, the processor 101 reads the member record 82R including themembership code by searching for the member database 82 with themembership code. Accordingly, the processor 101 obtains thecharacteristics information of sex and age from the member record 82R.If the characteristics information of the member can be obtained, theprocessor 101 proceeds to Act 6.

In Act 6, the processor 101 obtains flow line information of theshopper. That is, the processor 101 measures the movement direction(angle) of the cart C and the amount of movement (distance) in real timebased on a valued measured by the sensor group 108, and obtains the flowline information of the cart C, that is, the flow line information ofthe shopper by integrating the movement direction and the amount ofmovement. If the flow line information of the shopper can be obtained,the processor 101 proceeds to Act 7.

In Act 7, the processor 101 determines whether or not the commodity isinput to the shopping cart E. When the commodity is input to theshopping cart E, the data of the RFID tag attached to the commodity isread by the RFID reader 105. When the data of the RFID tag is not read,the commodity is not input to the shopping cart E. When the commodity isnot input to the shopping cart E (Act 7, NO), the processor 101 proceedsto Act 9.

When the data of the RFID tag is read, at least one commodity is inputto the shopping cart E. When the commodity is input to the shopping cartE (Act 7, YES), the processor 101 proceeds to Act 8.

In Act 8, the processor 101 obtains the commodity information of thecommodity that is input to the shopping cart E. That is, the processor101 detects the commodity code from the data of the RFID tag read by theRFID reader 105. Accordingly, the processor 101 obtains the commodityinformation such as the commodity name, the unit price, and commodityclassification of the commodity record 81R in which the commodity codeis set, by searching for the commodity database 81 with the commoditycode. At this time, when the data of two or more RFID tags are read bythe RFID reader 105, a process same as the above-described processperforms for each data. In this manner, if the commodity information ofthe commodity input to the shopping cart E can be obtained, theprocessor 101 proceeds to Act 9.

In Act 9, the processor 101 determines the presence or absence of theadditional information satisfying the provision condition. That is, theprocessor 101 searches for the additional information database 83 byusing the characteristics information obtained by the process of Act 5,the flow line information obtained by the process of Act 6, and thecommodity information obtained by the process of Act 8. Thecharacteristics information is the sex, the age, and the like of theshopper. The flow line information is a route to the sales place S1where the shopper moves. The commodity information is the information ofthe commodity input to the shopping cart E of the cart C by the shopper.The processor 101 determines the presence or absence of the additionalinformation record 83R in which the provision condition satisfying thecharacteristics information, the flow line information, and thecommodity information is set as Act 10. When a corresponding additionalinformation record 83R cannot be detected (Act 10, NO), the processor101 proceeds to Act 11.

In Act 11, the processor 101 outputs the image processed in the imageprocessing circuit 106, that is, the image in the reality space imagedby the camera 30 to the touch panel 107, and displays the output imageon a screen of the touch panel 107. Then, the processor 101 returns toAct 4.

Meanwhile, when the corresponding additional information record 83Rcannot be detected (Act 10, YES), the processor 101 proceeds to Act 12of FIG. 9. The provision condition may not satisfy the overall of thecharacteristics information, the flow line information, and thecommodity information. The processor 101 calculates a satisfaction levelfor each provision condition, and detects the additional informationrecord 83R in which the provision condition having the highestsatisfaction level is set. Alternatively, the processor 101 may detectthe additional information record 83R in which the closest provisioncondition is set with respect to the current position of the cart C, inconsideration with the provision condition and the sales position set inthe additional information record 83R.

In Act 12, the processor 101 obtains the sales position of the commodityto be recommended from a corresponding additional information record83R. Accordingly, the processor 101 proceeds to Act 13.

In Act 13, the processor 101 obtains the current position of the cart Cby the beacon signal received in the transmission and reception circuit104. In addition, the processor 101 specifies the movement direction ofthe cart C from a value measured by a sensor group. In this manner, ifthe sales position of the commodity to be recommended, the currentposition of the cart C, and the movement direction can be specified, theprocessor 101 proceeds to Act 14. The process of Act 12 and the processof Act 13 may be reversed.

In Act 14, the processor 101 decides a provision direction of theadditional information. That is, the processor 101 calculates angles ina horizontal direction and a vertical direction required for theshopper's sight line which is at the current position of the cart C andfaces the movement direction of the cart C to face to the sales positionof the commodity to be recommended, and sets the calculated angle as theprovision direction of the additional information. In this manner, ifthe provision direction of the additional information can be decided,the processor 101 proceeds to Act 15. Here, the processor 101 configuresa decision unit that decides a direction of providing the voiceinformation or the image information from a relationship between theposition of the cart C, and the movement direction and a position in thesales place S1 of the commodity to be recommended, to the cart C byprocesses of Act 12 to Act 14.

In Act 15, the processor 101 determines whether or not the additionalinformation record 83R includes the voice information. When the voiceinformation is not included (Act 15, NO), the processor 101 proceeds toAct 18. When the voice information is included (Act 15, YES), theprocessor 101 proceeds to Act 16.

In Act 16, the processor 101 localizes the virtual sound image in theprovision direction decided by a process of Act by controlling thevirtual sound image localization processing circuit 111. Furthermore,the processor 101 controls the virtual sound image localizationprocessing circuit 111 to reproduce voice of the voice information fromthe virtual sound image by the sound source 70 and a pair of left andright speakers 60L and 60R as Act 17. Then, the processor 101 proceedsto Act 18. With this, voice of the voice information that recommends thecommodity is heard in the direction of the commodity shelf R in whichthe commodity to be recommended is sold, to the shopper operating thecart C by the voice reproduced from the pair of left and right speakers60L and 60R. Here, the processor 101 configures a sound control unitthat controls the sound output unit to localize a sound image accordingto the voice information in a direction decided by performing processesof Act 16 and Act 17 in cooperation with the virtual sound imagelocalization processing circuit 111.

In Act 18, the processor 101 determines whether or not the additionalinformation record 83R includes the image information. When the imageinformation is not included (Act 18, NO), the processor 101 returns toAct 4. When the image information is included (Act 18, YES), theprocessor 101 proceeds to Act 19.

In Act 19, the processor 101 determines whether or not the provisiondirection decided in Act 14 is in a range of the image imaged by thecamera 30. In the present embodiment, it is possible to specify thecurrent position of the cart C and the movement direction in Act 13.Accordingly, a range of an imaged image can be specified from an angleof view of the camera that images the movement direction of the cart C.Here, the processor 101 configures a determination unit that determineswhether or not the direction decided by a process of Act 19 is in therange of the image imaged by the camera 30.

When the provision direction is outside the range of the imaged image(Act 19, NO), the processor 101 returns to Act 4. On the other hand,when the provision direction is in the range of the imaged image (Act19, YES), the processor 101 proceeds to Act 20.

In Act 20, the processor 101 specifies a display position of the imageby the image information to a position of the provision directiondecided in the process of Act 14. Accordingly, the processor 101synthesizes the image of the image information with the image of thereality space imaged by the camera 30 by controlling the imageprocessing circuit 106, as Act 21. At this time, the image processingcircuit 106 combines the image of the image information to be displayedat a position specified in a process of Act 20 of the image of thereality space. When the synthesis of the image is completed, theprocessor 101 proceeds to Act 22. Here, when the direction decided byperforming processes of Act 20 and Act 21 in cooperation with the imageprocessing circuit 106 is in the range of the image imaged by the camera30, the processor 101 configures a synthesis unit that synthesizes theimage of the commodity to be recommended with the decided direction ofthe image imaged by the camera 30.

In Act 22, the processor 101 outputs an image obtained by synthesizingthe image of the image information with the image processed by the imageprocessing circuit 106, that is, the image in the reality space imagedby the camera 30 on the touch panel 107, and causes to display the imageon a screen of the touch panel 107. Then, the processor 101 returns toAct 4. Here, the processor 101 configures an image control unit thatcontrols the synthesized image to be displayed on the touch panel 107,by a process of Act 22.

According to the present embodiment, the voice of the voice informationfor recommending the commodity is heard in the direction of thecommodity shelf R in which the commodity to be recommended is sold tothe shopper turning around the sales place S1 by operating the cart C.Accordingly, the shopper can easily specify whether the commodity to berecommended is displayed on the commodity shelf R in which direction.

In addition, according to at least one embodiment, the image imaged bythe camera 30, that is, the image indicating the reality space in themovement direction of the cart C is displayed on the touch panel 107 ofthe cart terminal 10 in real time. Accordingly, when the voice of thevoice information for recommending the commodity from, for example, thefront, the shopper can confirm the commodity shelf R or the like in adirection which can hear the voice from the image of the touch panel107. In addition, when the voice of the voice information forrecommending the commodity can be heard from, for example, the back, bysetting the cart C in a direction in which the voice is heard, it ispossible to confirm the commodity shelf R or the like in a direction inwhich the voice is heard from the image of the touch panel 107.

Moreover, according to the present embodiment, when the image in thereality space imaged by the camera 30 includes the commodity shelf R inwhich the commodity to be recommended is sold, the image of thecommodity to be recommended is displayed in the direction of thecommodity shelf R. Accordingly, the shopper can visually confirm adirection in which the commodity to be recommended is sold.

In addition, according to at least one embodiment, the shopper onlymoves the cart C from the cart storage place S2 to the sales place S1such that the cart terminal 10 automatically enters the recommendationmode. Accordingly, the shopper can obtain information on the recommendedcommodity without selecting the recommendation mode manually. Inaddition, in a store side, since the cart terminal 10 is automaticallyset to the recommendation mode and the information of the commodity tobe recommended to the shopper can be automatically, which leads to salespromotion.

In the above-described embodiments, the following at least onemodification embodiment is conceivable. In the at least one embodiment,the processor 101 determines whether or not the provision directiondecided in Act 14 is in the range of the image imaged by the camera 30,in Act 19 of FIG. 9. Accordingly, when the decided provision directionis in the range of the image, the processor 101 specifies the displayposition of the image based on the image information is determined to aposition of the decided provision direction in Act 20. Regarding thispoint, processes of Act 19 and Act 20 may be omitted. That is, when theadditional information record 83R includes the image information (Act18, YES), the processor 101 synthesizes the image of the imageinformation with the image of the reality space imaged by the camera 30.Even if such a configuration is adopted, the shopper has an effect thatit is visually confirmed whether the commodity to which information isprovided by voice is what kind of the commodity.

In at least one embodiment, the cart terminal 10 for providing theinformation of the commodity to be recommended to the shopper shoppingby using the cart C is exemplified as the information providingapparatus. The information providing apparatus is not limited to theabove-described cart terminal 10. For example, provision of theinformation by the voice is very effective for a visually impaired.Accordingly, for the shopper with visual impairment, it is also possibleto realize the cart terminal notifying voice information such as whetherthe commodity in a shopping list registered by the shopper is sold onthe commodity shelf in which direction and whether there is a person inthe vicinity of the cart C, as a modification example of the embodiment.

In addition, the moving object is not limited to the cart C. Forexample, a forklift used in a warehouse or the like may be adopted asthe moving object. In this case, for a user operating the forklift, itis conceivable to notify that a desired article is placed by voice in ashelf direction where the desired article is placed, it is possible tonotify. In addition, it is also possible to configure the moving objectas a person such as the shopper and to configure a headphone, a tabletterminal with speaker built in, and the like used by this person as theinformation providing apparatus.

Transfer of the cart terminal 10 as the information providing apparatusis generally performed in a state where a program such as the controlprogram is stored in the main memory 102 or the auxiliary storage device103. However, the embodiment is not limited thereto, in a state wherethe program is not stored in the main memory 102 or the auxiliarystorage device 103, the program may be transferred. Accordingly, in thiscase, the control program or the like individually transferred from thecart terminal 10 is written to a writable storage device of the cartterminal 10 in accordance with an operation of a user or the like. Theabove-described another control program may be rewritten. The transferof the control program and the like can be performed by recording on aremovable recording medium or by communication via a network the controlprogram. The recording medium may be in any form as long as therecording medium can store a program such as a CD-ROM and a memory cardand the device can be read. In addition, a function obtained byinstalling or downloading a program may be one which realizes thefunction in cooperation with an operating system (OS) or the like insidethe device.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel methods and systems describedherein may be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the methods andsystems described herein may be made without departing from the spiritof the inventions. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fall within thescope and spirit of the inventions.

What is claimed is:
 1. An information providing apparatus comprising: atleast one computer configured to: detect a position and a movementdirection of a moving object in an area in which an article,corresponding to information that is provided, is disposed; and decide adirection in which the information is provided to the moving object froma relationship between the position of the moving object, the movementdirection of the moving object, and a position in the area of thearticle; and a sound control circuit configured to perform virtual soundimage localization processing and control a sound output attached to themoving object using a plurality of speakers, the sound output beinglocalized according to the information in the direction decided by theat least one computer.
 2. The apparatus according to claim 1, whereinthe at least one computer is configured to cause an image imaged by animaging device attached to the moving object to be displayed on an imagedisplay attached to the moving object.
 3. The apparatus according toclaim 2, wherein the at least one computer is configured to synthesizean image of the article with the image imaged by the imaging device,wherein the at least one computer is configured to cause the imagesynthesized to be displayed on the image display.
 4. The apparatusaccording to claim 3, wherein the at least one computer is configured todetermine whether or not the direction decided is in a range of theimage imaged by the imaging device, wherein the at least one computer isconfigured to synthesize the image of the article with the directiondecided when the direction decided is in the range of the image imagedby the imaging device.
 5. The apparatus according to claim 1, whereinthe at least one computer is configured to check whether or not themoving object detected enters an inside region of the area from a regionoutside of the area, wherein the at least one computer is configured toswitch a mode to a mode of providing the information under a conditionthat the moving object enters the inside region of the area.
 6. Theapparatus according to claim 1, wherein the plurality of speakers aredirectional speakers.
 7. The apparatus according to claim 1, wherein theimaging device includes a camera.
 8. The apparatus according to claim 1,wherein the moving object is a least one of a cart or a forklift.
 9. Aninformation providing method comprising: detecting a position and amovement direction of a moving object in an area in which an article,corresponding to information that is provided, is disposed; deciding adirection in which the information is provided to the moving object froma relationship between the position of the moving object, the movementdirection of the moving object, and a position in the area of thearticle; and performing virtual sound image localization processing andcontrolling a sound output attached to the moving object using aplurality of speakers, the sound output being localized according to theinformation in the decided direction.
 10. A cart comprising: a frame; acart terminal connected to the frame; a left speaker electricallyconnected to the cart terminal; and a right speaker electricallyconnected to the cart terminal, the cart terminal having at least onecomputer configured to: detect a position and a movement direction of amoving object in an area in which an article, corresponding toinformation that is provided, is disposed; and decide a direction inwhich the information is provided to the moving object from arelationship between the position of the moving object, the movementdirection of the moving object, and a position in the area of thearticle; and a sound control circuit configured to perform virtual soundimage localization processing and control a sound output attached to themoving object using a plurality of speakers, the sound output beinglocalized according to the information in the direction decided by theat least one computer.
 11. The cart according to claim 10, wherein theat least one computer is configured to cause an image imaged by animaging device attached to the cart to be displayed on a display of thecart terminal.
 12. The cart according to claim 11, wherein the at leastone computer is configured to synthesize an image of the article withthe image imaged by the imaging device, wherein the at least onecomputer is configured to cause the image synthesized to be displayed onthe display.
 13. The cart according to claim 12, wherein the at leastone computer is configured to determine whether or not the directiondecided is in a range of the image imaged by the imaging device, whereinthe at least one computer is configured to synthesize the image of thearticle with the direction decided when the direction decided is in therange of the image imaged by the imaging device.
 14. The cart accordingto claim 10, wherein the at least one computer is configured to checkwhether or not the moving object detected enters an inside region of thearea from a region outside of the area, wherein the at least onecomputer is configured to switch a mode to a mode of providing theinformation under a condition that the moving object enters the insideregion of the area.
 15. The apparatus according to claim 10, wherein theplurality of speakers are directional speakers.
 16. The apparatusaccording to claim 10, wherein the imaging device includes a camera.